Watermarking systems and methods

ABSTRACT

Various improvements and applications for digital watermarking technology are detailed. One concerns techniques for making watermarks resistant to malicious attacks. Another involves using digital watermarking with ID cards and credentials, such as a watermarked driver&#39;s license conveying a user&#39;s cryptographic PKI. Still another uses digital watermarks in connection with automated compliance audits for corporate users of electronic content. A variety of other technologies are also detailed.

RELATED APPLICATION DATA

This application claims priority to provisional application 60/445,478,filed Feb. 5, 2003.

FIELD OF THE INVENTION

The present disclosure memorializes various improvements relating todigital watermarking.

BACKGROUND

Digital watermarking is the science of encoding physical and electronicobjects with plural-bit digital data, in such a manner that the data isessentially hidden from human perception, yet can be recovered bycomputer analysis. In physical objects, the data may be encoded in theform of surface texturing, or printing. Such marking can be detectedfrom optical scan data, e.g., from a scanner or web cam. In electronicobjects (e.g., digital audio or imagery—including video), the data maybe encoded as slight variations in sample values. Or, if the object isrepresented in a so-called orthogonal domain (also termed“non-perceptual,” e.g., MPEG, DCT, wavelet, etc.), the data may beencoded as slight variations in quantization values or levels. Thepresent assignee's U.S. Pat. Nos. 6,122,403 and 6,614,914 areillustrative of certain watermarking technologies.

Watermarking can be used to tag objects with a persistent digitalidentifier, and as such finds myriad uses. Some are in the realm ofdevice control—e.g., tagging video data with a do-not-copy flag that isrespected by compliant video recorders. (The music industry's SecureDigital Music Initiative (SDMI), and the motion picture industry's CopyProtection Technical Working Group (CPTWG), are working to establishstandards relating to watermark usage for device control.) Otherwatermark applications are in the field of copyright communication,e.g., indicating that an audio track is the property of a particularcopyright holder.

Other watermark applications encode data that serves to associate anobject with a store of related data. For example, an image watermark maycontain an index value that serves to identify a database recordspecifying (a) the owner's name; (b) contact information; (c) licenseterms and conditions, (d) copyright date, (e) whether adult content isdepicted, etc., etc. (The present assignee's MarcCentre service providessuch functionality.) Related are so-called “connected content”applications, in which a watermark in one content object (e.g., aprinted magazine article) serves to link to a related content object(e.g., a web page devoted to the same topic). The watermark canliterally encode an electronic address of the related content object,but more typically encodes an index value that identifies a databaserecord containing that address information. Application Ser. No.09/571,422 details a number of connected-content applications andtechniques.

One problem that arises in many watermarking applications is that ofobject corruption. If the object is reproduced, or distorted, in somemanner such that the content presented for watermark decoding is notidentical to the object as originally watermarked, then the decodingprocess may be unable to recognize and decode the watermark. To dealwith such problems, the watermark can convey a reference signal. Thereference signal is of such a character as to permit its detection evenin the presence of relatively severe distortion. Once found, theattributes of the distorted reference signal can be used to quantify thecontent's distortion. Watermark decoding can then proceed—informed byinformation about the particular distortion present.

The assignee's U.S. Pat. Nos. 6,408,082 and 6,614,914 detail certainreference signals, and processing methods, that permit such watermarkdecoding even in the presence of distortion. In some image watermarkingembodiments, the reference signal comprises a constellation ofquasi-impulse functions in the Fourier magnitude domain, each withpseudorandom phase. To detect and quantify the distortion, the watermarkdecoder converts the watermarked image to the Fourier magnitude domainand then performs a log polar resampling of the Fourier magnitude image.A generalized matched filter correlates the known orientation signalwith the re-sampled watermarked signal to find the rotation and scaleparameters providing the highest correlation. The watermark decoderperforms additional correlation operations between the phase informationof the known orientation signal and the watermarked signal to determinetranslation parameters, which identify the origin of the watermarkmessage signal. Having determined the rotation, scale and translation ofthe watermark signal, the reader then adjusts the image data tocompensate for this distortion, and extracts the watermark messagesignal as described above.

With the foregoing by way of background, the specification next turns tothe various improvements. It will be recognized that these improvementscan typically be employed in many applications, and in variouscombinations with the subject matter of the patent documents citedherein.

DETAILED DESCRIPTION 1. Template-Based Watermark Attacks andCountermeasures

Attacking the subliminal calibration signal found in some watermarks(e.g., the template signal) is one way of attacking a watermark. (Oneform of template is shown in the assignee's U.S. Pat. No. 6,408,082.)

To identify the template, some have proposed averaging many images sothe images cancel each other while the template sums, thus making thetemplate easier to identify (i.e. collusion). If the template were tocancel—rather than sum—when many images are averaged, it will be morerobust to identification.

The idea is to add the template to some images and subtract the templatefrom other images. The subtracted template may be more difficult tolocate for authorized purposes since it is harder to find holes. Anotherapproach would thus be to rotate the template, or shift the template inphase randomly throughout different images so that during averagingimages the template won't sum. This approach is very useful if theanalysis of the template is rotation invariant, such as used todetermine the orientation and scaling of a watermark tile since thedetector requires nothing extra to detect the collusion robust templateversus a non-collusion robust template.

Yet another approach is to slightly scale the template. (The '914technique is robust to both rotation and scaling.)

While the foregoing techniques address collusion attacks based onavailability of several watermarked images, other attacks may be basedon a single image, since the template signal is replicated many timeswithin a single image. Such attacks may be redressed by obscuring thetemplate, e.g., by designing it to only show up in an obscure, alternatedomain (countless of which may be devised). Or to hide the templatebehind a key—using an invariant keying method that doesn't requireadditional search over different scale and rotation values to locate.(Such techniques are further detailed in published US patent application20030012401.)

It will be recognized that such techniques can be applied to templatesignals of various forms. Such signals generally comprise collections ofembedded features that can be used to facilitate computation of thegeometric distortion of the embedded signal. An example is aconstellation of peaks in a particular transform domain (e.g., fourieror autocorrelation). It will be recognized that such a collection ofembedded features may result from an explicit template signal, or may bea desired consequence formed by patterning of the watermark (e.g., byregular tiling).

Related methodology helps perform preliminary detection of a templatesignal in the Fourier domain. It has three stages. First, find localmaxima on half of the Fourier magnitude array. Then use the 90-degreerotation symmetry of the template to eliminate most of the local maximain a quadruple, where in this stage certain tolerance is added. Third,check each pair of the left maxima in the quadruple to see if the anglebetween them and the ratio of their radial distances to the origin makethem a pair of points on our template or not. If they are, what scalefactor and orientation angle of the template in this case. After runningthrough the total of about 50 maxima (for 128×128 block), accumulatedcount on a particular orientation and scale factor will indicate whatthe orientation and scale factor of the gird is when there is a templatesignal. A threshold is used to judge if there is a template signal ornot. (This methodology avoids log-polar re-mapping, as is required bysome other approaches.) Because this method is based on the ratios ofthe radial distances to the origin of any pair of template points, avariety of advantageous template designs may be achieved. (Additionaldetails on related technology are to be found in application Ser. No.10/302,753, filed Nov. 22, 2002.)

When using the template methodology just described, the template can berotated differently in each image or each tile, and still be detectedwith similar effort as long as the template is symmetric when rotated by90 degrees, because the symmetric template method is based upon thisrotational symmetry of the template. By rotating the template in eachimage or tile, it shouldn't add for collusion attack as the peaks willbe in different frequencies and phase.

2. MPEG Digital Watermark Detector with Drift Compensation

For video content embedded in baseband that has been compressed, thewatermark detector can improve its detection by taking the inverse ofdrift compensation into account. Specifically, the detector could makean estimate of the watermark noise sequence that was included in thedifference coefficients in a predictive frame and subtract this estimatefrom the difference to remove inappropriately placed watermark data. Byremoving this inappropriate watermark data, the image can be renderedwithout the interference, thus looking better, and the watermark can bedetected with less interference.

More specifically, for the detector to remove the inappropriatewatermark data, the watermark payload and embedding key, such as PNsequence, need to be known. In addition, the watermark data that was notembedded with a large local gain is probably not included in thedifference of the predictive frame and should be ignored. Thus, for thedetection process where only the key (e.g. PN sequence) is known, thepayload can be estimated by detecting the payload without removing theinappropriate watermark data, and then from this estimate removing theinappropriate data and re-detecting the payload. This process can bedone iteratively until the payload meets detection threshold criteria.

In addition, since the inappropriate watermark data can be removed, thevideo can be rendered with less interference from the watermark. If thewatermark payload is constant through the video, the inappropriatewatermark data can be removed from each frame without the iterativeprocess once the payload is known. If the video can be detectednon-linearly (i.e. detect on frames not related to the current framebeing rendered, such as with a DVD), the detector can find key frames(e.g. I-frames in MPEG video) to detect the payload, especially keyframes that have characteristics that enable the watermark signal to beembedded with a lot of gain, such as noisy I-frames in MPEG video withPN sequence based watermarking techniques.

Even if the content has been compressed more than once, this techniqueshould help for the last compression.

3. Digital Identity Business Architecture

Consumers are used to physical identification (ID) cards beinginexpensive.

This fact arises since these cards are usually government issued, andthe government is not looking to make money and can subsidize the IDwith tax money. However, For businesses to create digital identity,there must be money for the business to survive.

Asking a high price for the digital identity is usually not acceptable,since the consumer has to pay or the card price is related to otheruses. For example, drivers licenses (DLs) with watermarks can be used tophysically identify the driver of a car and can be used as a digitalidentity, but the DL cannot double in price or voters would not acceptthe change. However, if the watermarked DL is used for every onlinetransaction, the company providing the watermark technology should becompensated for the value of its system.

A novel compensation method and architecture is to share in thee-commerce transaction with the provider of the digital identity card orenabling technology, as currently done with credit cards. The differenceis that the sharing cost is not related to providing credit, and thesystem uses a different architecture than the credit card system, whichwithholds some of the payment to the retailer.

Before an item is purchased in an online environment, the consumer'sidentity must be electronically confirmed since there is no retailer tolook at the person and, optionally, compare their name and picture to acredit card. There are several ways to prove identity, such as providinga digital certificate via registration authority or providing awatermarked ID card, such as drivers license, to secure softwarecontrolling a PC camera.

When any of these digital identity methods are provided, the digitalidentification system can require e-tailer (i.e. electronic retailer) toprovide a fixed fee and/or fractional amount of the sale enabled by thedigital identification system.

A preferred embodiment employs the following architecture. Theverification of digital identity happens at the retailer and/or via acomputer provided by the digital identification service provide—probablyvia a secure reader provided by the digital identity service provider.After this verification, a computer belonging to the digitalidentification service provider securely communicates with an e-tailercomputer for the transfer of funds or credit for each transactionenabled by the digital identification.

The location of these two computers or systems is irrelevant. The factthat the digital identification computer is securely controlled by thedigital identification service provider and that the e-tailer's computeris securely controlled by the e-tailer is the most important. In fact,these two “computers” could be part of one computer whose memory,processing and hard drive is securely divided, such as via MicrosoftPalladium architecture.

Alternatively, if the transaction uses a credit card the credit cardcompany may transfer the funds to the digital identification serviceprovider. The architecture would be the same as above, except the creditcard company replaces the e-tailer.

In a similar alternative embodiment, the credit card may have thedigital identification as part of it, such as a watermark on the creditcard or smart ID chip inside the credit card. In this case, the creditcard company can still send the digital identity service provider thefee in the architecture as described above. However, the credit cardcompany could withhold a larger amount than currently withheld duringtransactions, and share a percentage of the withholdings in a periodic,i.e. monthly, check with the digital identity service provider.

4. Database Format for Identifying Distribution Object

When an object is identified during distribution with an identificationtechnology, an optimal database structure for storage of information isshown below. The database can be stored locally, such as if theidentification method or similar item on the object has memory storage,such as a smart ID chip. The database can also be stored remotely, inany distributed format.

The database format includes fields for:

Product ID

Product Name

Product Description, i.e. what is contained

Raw Material IDs (linked list)

Manufacturer

Manufacture Contact Information, with name, phone, email, etc.

Recipient

Recipient Contact Information, with name, phone, email etc.

Origin

Destination

Destination Path (linked list with all desired locations)

Current Location (linked list with all past locations)

Inspections (linked list with all past locations)

Next Destination

Type/Risk

The linked list fields of the database will grow over time. Any of theselists can only include the information during the most recentinspection, or past few inspections, if it is desired for the databaseto be of fixed size. The IDs in the list can be proprietary or relate toany of the many standards, such as ePC (electronic product code), ISCI(Industry Standard Coding Identification), UPC, etc.

This is the beginning of such a list and format. Alternatively, thedatabase can store the fields dedicated in PML and/or PDML, and theselanguages can be easily and dynamically created from the database, andthe database can be easily and dynamically created from these languages.

5. Public Key Dissemination Using Watermarks

Documents and other watermarkable-objects (physical or virtual) canconvey public keys. Thus, for example, a person's public key identifiermay be digitally watermarked on her driver's license or business card.The PKI infrastructure has been developing for years, it is in everybrowser and it continues to mature. The issue with PKI isn't that thetechnology doesn't work; it's that last mile problem of having peopleget and understand the private keys and certificates. The way that mostpeople best understand PKI is when they have something like a card thatrepresents their identity and prompts them to show it (something theyhave) to demonstrate their identity. It's hard to get this conceptacross with software tokens but straightforward with a card becausepeople do this all the time.

A cryptographic system could be integrated with common Internet browsersoftware in a similar way that a smartcard would (although the keyswould still have to be stored in software). The advantage of this is youcould then hook into all of the PKI technology that exists out there forthings like digital signatures and secure access via a printed card witha watermark.

A person could take an identity card home, present it in front of theirwebcam, and register their card—which behind the scenes generates theappropriate key pair on their machine (or perhaps on a key server) andcreates a digital certificate.

There may also be a signing scheme where a person with one of thesecards could digitally sign and print something like an encrypted hashinto the image. And then a receiver of the document could receive it,retrieve the embedded id and encrypted hash, look up the digitalcertificate using the embedded id, and validate the signature.

A magnetic stripe card could do this as well, as a smart card orwatermarked object, or a token conveying an RFID chip or bar code.

Such technology also addresses the problem of locating the public key ofsomeone who has sent me information, such as an email. Watermarkedbusiness cards can link to the public key, along with a software programthat automatically enters my contact information into another program,such as Microsoft's Outlook e-mail and personal information managementsoftware.

6. Watermark-Based Copyright Compliance Audits

Watermarks could be used for doing corporate compliance audits. Ideawould be that content owners or others would watermark content (audio,video, images, text documents) and thencorporations/organizations/government agencies could then run complianceaudits (similar to the manual software audits that are done of somecompanies today but using watermarks to automate the process).

Watermark detection toolkits could be deployed on a corporate intranetto look at servers, databases and desktop PC's to see if they have anywatermarked content that is identified as copyrighted and report backits location and what it is.

Could also be deployed at the corporate firewall to: filter content out,keep content in, and report on what is being viewed/downloaded over/ontothe corporate internet. This could be used by companies as a riskmanagement tool to help them control copyrighted material form beinginappropriately used/stored/copied on their internet.

The same techniques can also employ fingerprinting to identify objects.

7. Metallic Ink on Identification Cards

Magnetic inks or inks with metallic objects may be used to give aprinted image on a card the ability to be read by an RF reader. A lessexpensive RFID card may thus be produced with the print providing theresponse.

There is a class of detectors that are used for magnetic imaging offeatures on currency, like the ones used by the Federal Reserve Bank forcheck and banknote sorting, which are able to recover small amounts ofdata (patterns really) from structures to determine whether a bill isgenuine or counterfeit. Some examples are shown on the website of SanDiego Magnetics, at the sdmagnetics.com domain.

A related concept is to form a diffraction grating on an identity cardusing metallic, magnetic, or other inks having desired diffractionqualities at the frequencies of interest. To keep the physicaldimensions small enough for use on a card, energy in the microwave, IR,visible, or UV wavelengths may be required.

The diffraction grating may be formed on one side of the card, with theother serving as a uniform reflective plane (e.g., metal or otherwise).Reflective media may also be embedded within the card. The goal is tomake the slit appear bigger than it is to provide the diffractionpattern for RF, since RF is much larger wavelength than visible light.

To provide a comprehensive disclosure without unduly lengthening thisspecification, the patents and applications cited above are incorporatedherein by references.

Having described and illustrated the subject technologies with referenceto illustrative embodiments, it should be recognized that the inventionis not so limited.

For example, while the detailed description focused on digitalwatermarks, other techniques can be used as well depending on theparticular application context (e.g., VBI, digital fingerprints, headermeta data, bar codes, glyphs, RF IDs, mag stripes, smart cardtechnology, etc.).

The implementation of the functionality described above (includingwatermark decoding) is straightforward to artisans in the field, andthus not further belabored here. Conventionally, such technology isimplemented by suitable software, stored in long term memory (e.g.,disk, ROM, etc.), and transferred to temporary memory (e.g., RAM) forexecution on an associated CPU. In other implementations, thefunctionality can be achieved by dedicated hardware, or by a combinationof hardware and software. Reprogrammable logic, including FPGAs, canadvantageously be employed in certain implementations.

It should be recognized that the particular combinations of elements andfeatures in the above-detailed embodiments are exemplary only; theinterchanging and substitution of these teachings with other teachingsin this and the incorporated-by-reference patents/applications are alsocontemplated.

1. In a method that includes encoding one or more content objects with asteganographic digital watermark, the encoding including embedding acollection of features that can be used to facilitate computation ofgeometrical distortion of the object after encoding, the geometricdistortion including rotation, an improvement including step for makingthe collection of features resistant to collusion attack.
 2. The methodof claim 1 wherein said step includes adding said collection of featuresin some of said objects, and subtracting said collection of featuresfrom other of said objects.
 3. The method of claim 1 wherein said stepincludes embedding said collection of features at a first scale in afirst object, and embedding said collection of features at a second,different scale in a second object.
 4. The method of claim 1 whereinsaid step includes embedding said collection of features at a firstorientation in a first object, and embedding said collection of featuresat a second, different orientation in a second object.
 5. The method ofclaim 1 wherein said step includes obscuring said collection of featuresby designing same to become apparent only in an alternate domain.
 6. Anobject produced by the process of claim 1.